Method for providing clinical support for surgical guidance during robotic surgery

ABSTRACT

A method for providing clinical support for surgical guidance during robotic surgery.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority of U.S. provisionalapplication No. 62/898,880, filed 11 Sep. 2019, the contents of whichare herein incorporated by reference.

BACKGROUND OF THE INVENTION

The present invention relates to robotic surgical procedures and, moreparticularly, a computer-implemented method for providing clinicalsupport for surgical guidance during robotic surgery.

MRIs contain crucial information for surgical planning. In fact, the useof the three-dimensional (3D) models obtained from multiparametric MRI(mpMRI) has become the standard of care in the United Kingdom and inEurope for staging of prostate cancer. In the United States, theNational Clinical Cancer Network has also added the use of mpMRI in thescreening and staging of prostate cancer.

The surgeon's skill level to interpret mpMRI, however, has not kept pacewith the proliferation of mpMRI of the prostate. Generally speaking,surgeons have a limited understanding of intra-operative prostateanatomy correlated with mpMRI of the prostrate since mpMRI is new forurologist and there is no residency training to interpret MRIs, andcurrently no way to actively use mpMRI information during surgery. As aresult, the surgeons are unable to use the valuable information toimprove or tailor their surgical approach; for instance, to avoidcutting through tumors during surgery.

Prostate cancer is the most common male malignancy and one man out ofseven will be diagnosed with prostate cancer during his lifetime.Radical prostatectomy, that represents the surgical extirpation of thecancer, is performed by means of robotic surgery in the vast majority ofthe hospitals in the USA.

In 2009 the FDA cleared the first device that could use information froman MRI to guide a biopsy using technology that merges the ultrasound andMRI data sets to guide the prostate biopsy. There is level 1 evidence(2019) that this new approach is superior to the prior ultrasound onlybiopsy approach. In 2013, the commercial systems became available in theUnited States. The pathology results and 3D data from these biopsyapproaches can be stored and reviewed prior to the surgical treatmentsusing customized workstations. A key component is that these data pointswere not available until 2013 and that each biopsy core is tracked andrecorded in 3D space. This data, which includes cancer location andextension, is then matched back to the mpMRI of the prostate and a 3Dmap of a patient's prostate and critical structures can be created.

One of the goals of the surgical treatment for prostate cancer (PCa),along with oncological control, is to provide patients with optimalquality of life. In an effort to achieve that, surgeons must attempt a“conservative” dissection allowing for maximal preservation of thestructures surrounding the prostate, including the neurovascularbundles. The extent of the dissection will ultimately influence urinarycontinence and erectile function.

Considering the fact that surgeons walk on a fine line trying to balancethe risk of extra-prostatic disease and the risk of resecting throughtumorous tissue, the frequency of Post-Surgical Margins (PSM) is higherin men diagnosed with prostate cancer relative to those diagnosed withother malignancies. For instance, compared to the most common femalemalignancy, i.e., breast cancer, the PSM rate in surgically treated PCais almost four times higher.

In short, the urologist requires the expertise of a radiologists todecipher MR Images to help them tailor their surgical plan, yet thecurrent information that urologist require during surgery areunfortunately available in multiple different, disparate softwaresystems within the hospital. These systems may provide the information,but most surgeons lack the skill and expertise to maximize surgicalperformance. In other words, the information to aid these surgeons isavailable; however, the information is not accessible during roboticsurgery.

The utilization of 3D models at the time of surgery can provide surgeonswith visual information denoting the location of the tumor—offering thepotential not only to tailor to each patient the best surgical approachbut also to reduce PSMs, which occur anytime a surgeon cuts through atumor and leaves a piece of malignant tissue inside the human body.Also, there is currently no computer-based or software-based solutionallowing clinical/imaging data to be available to surgeons during realtime surgery on the robotic console.

As can be seen, there is a need for a computer-implemented method forproviding clinical support for surgical guidance during robotic surgery.The present invention will draw information from the diagnosticradiology studies, 3D MR US fusion guided biopsy data, clinicalvariables and allow it to be accessed during surgery through a softwareapplication interface that broadcasts visual information into therobotic surgery platform and provides a controller for manipulating datain real-time. Thereby, the present invention integrates 3D modulesobtained from mpMRIs in the robotic console at the time of surgery toassist with predicting extra-prostatic extension; defining large tumorvolume abutting the capsule; identifying the location of the tumor;optimizing nerve dissection; and providing surgeons with a visual toolduring the dissection phase of robot-assisted radical prostatectomy.

The data will be accessed, and never stored, on a portable device, suchas, but not limited to, a smartphone, tablet or laptop, connected to thehealthcare internal network. The data will be available on the device;this device will be streaming the information to the robotic console forenhancing surgical planning and improving surgery performance. The datawill not be modified; rather, the sole role of the portable device is todisplay data. Images and patient data will never be modified duringsurgery. Under no circumstances will the process embodied in the presentinvention interfere with the functions of the robotic system. The datawill be displayed in the screen of the robotic console.

One example of a medical robotic system is the daVinci® Surgical Systemfrom Intuitive Surgical, Inc., of Sunnyvale, Calif. The daVinci® systemincludes a surgeon's console, a patient-side cart, a high performance3-D vision system, and Intuitive Surgical's proprietary EndoWrist™articulating instruments. In this system, the TilePro™ function, whichis a default function of the surgical console allows to split the screenin parts for a multi-input display of information.

The aforementioned data will be displayed in one of the parts of thescreen of the console and never superimposed to the images thatoriginate from the patient's body. If using the the daVinci® system, thedata will be displayed using the TilePro™ function; if using anotherrobotic system, the data will be displayed through their split screenfunction.

Considering robotic surgery is an inherently computer-based field, thecomputer-implemented method for providing clinical support for surgicalguidance during robotic surgery would improve this technological fieldand related fields such as robotic consoles, surgical interfaces and thelike.

The present invention applies also to robotic surgical procedures thatare performed on any parenchymal or hollow organ or robotic surgicalprocedures in general whenever preoperative imaging planning is deemedappropriate by the treating physician.

SUMMARY OF THE INVENTION

In one aspect of the present invention, a system for providing clinicalsupport for a surgery includes the following: a processor; a displaydevice to display one or more real time images originating from aselected patient; and a memory having computing device-executableinstructions that, when executed by the processor, cause the processorto implement: a communications interface for accessing a magneticresonance imaging data and an electronic health record data for theselected patient over a network, wherein said data is accessed but neverstored; a user interface for displaying and interacting with thecommunications interface; and a generation module for broadcasting theuser interface on the display device juxtaposed to said one or more realtime images, whereby the user interface is never superimposed on the oneor more real time images.

In another aspect of the present invention, the system for providingclinical support for a surgery a processor; a surgical console having todisplay one or more real time images originating from a selectedpatient; and a memory having computing device-executable instructionsthat, when executed by the processor, cause the processor to implement:a communications interface for accessing a magnetic resonance imagingdata and a fusion biopsy data set for the selected patient over anetwork, wherein said data is accessed but never stored, and wherein themagnetic resonance imaging data and the electronic health record datacomprise a preoperative imaging plan; a user interface for displayingand interacting with the communications interface; and a generationmodule for broadcasting the user interface on the surgical console insuch a way that the user interface is juxtaposed to said one or morereal time images, wherein the magnetic resonance imaging data comprisesa three dimensional model configured to be rotatable about athree-dimensional axis, and wherein the three-dimensional modelconfigured to provide zoom functionality and highlighting functionality,whereby the user interface is never superimposed on the one or more realtime images, wherein the user interface is displayed along one screen ofa split-screen functionality generated by the generation module, andwherein the display device has an integrated broadcasting device forreceiving the user interface.

These and other features, aspects and advantages of the presentinvention will become better understood with reference to the followingdrawings, description and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The sole FIGURE is a flow chart of an exemplary embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

The following detailed description is of the best currently contemplatedmodes of carrying out exemplary embodiments of the invention. Thedescription is not to be taken in a limiting sense, but is made merelyfor the purpose of illustrating the general principles of the invention,since the scope of the invention is best defined by the appended claims.

Referring now to the sole FIGURE, the present invention may include atleast one computer with a user interface. The computer may include atleast one processing unit coupled to a form of memory. The computer mayinclude, but not limited to, a microprocessor, a server, a desktop,laptop, and smart device, such as, a tablet and smart phone. Thecomputer includes a program product including a machine-readable programcode for causing, when executed, the computer to perform steps. Theprogram product may include software which may either be loaded onto thecomputer or accessed by the computer. The loaded software may include anapplication on a smart device. The software may be accessed by thecomputer using a web browser. The computer may access the systemicsoftware application via the web browser using the internet, extranet,intranet, host server, internet cloud and the like.

A method of using the present invention may include the following. Thesystemic software application enables the user-medical professional toselect a patient to be queried through the present invention's ‘Patientfor Surgery’ modality. The systemic software application then queriesthe Application Programming Interfaces (API) for all hospital-basedsystems, such as Dynacad, IntelliSpace Precision Medicine, ComputerAided diagnostic systems, UroNav, fusion biopsy platforms, and the likefor electronic health record data for the selected patient. The systemicsoftware application may be adapted to extract relevant data pointswhich can include, but are not limited to, demographic, lab values, and3D data from Dynacad and electronic medical records from theaforementioned platforms. The systemic software application isconfigured to be HIPAA and HITRUST compliant.

The data will then be retrieved and then displayed for the user on adisplay screen for selective manipulation for surgical planning via thesystem software application. Specifically, in certain embodiments,during manipulation the data is broadcast/transmitted to the roboticconsole via the robotics DVI port or broadcast/transmitted directly to arobotic console having an integrated broadcasting device. The user willbe able to see current demographic, prostate cancer information andselect a Surgical Planning Tool modality. The Surgical Planning Toolincludes a three-dimensional model of the prostatic gland and theprostate tumor that can be rotated on the three-dimensional axis (x, yand z) and can be zoomed in or out. The orientation can be adjusted topresent surgical views to aid with bladder neck dissection, nervesparing approach, and urethral sphincter mobilization. In the model,area(s) with the tumor will be identifiable in part through a differentcolor, relative to the “healthy” prostatic parenchyma, i.e. area of theprostate devoid of tumor. Users are able to highlight different areas ofrisk, e.g., the urethra, rectum, previous positive biopsy cores, areasof possible extra-prostatic extension, seminal vesicles, vasa deferentiaand other anatomical structures.

The content of the systemic software application, that is displayed onthe screen of the smartphone, will be broadcast to the robotic surgicalplatform. The broadcasting will be realized by means of, but not limitedto, Google Chromecast, Amazon Firestick and Apple TV, S-video, DVI,HDMI.

The systemic software application enables a server-based softwaresolution accessing multiple data systems while organizing and presentingthe data to the surgeon in the operating theatre by way of aconsolidation module coupling a systemic user interface and the roboticconsole. The systemic software application may be coded in 10S, Android,React native and or web-based platform. The systemic softwareapplication may act as a conduit for information between hospital basedEMR, computer aided diagnostic systems, and biopsy platforms whilecombining the information for transmission, display, orientation duringrobotic surgery on the robotic surgery platform. The systemic softwareapplication could be used for other surgical navigation using 3D imagingfor surgical planning.

The system includes at least one computing device having a processor anda memory. The memory includes software in the form of computingdevice-executable instructions that, when executed by the processor,cause the processor to implement: a communications interface, a userinterface, and a consolidation module.

The computing device is at least the processor and the memory. Thecomputing device may include a smart phone, a tablet computer, a laptop,a desktop, and the like. The computing device may execute on anysuitable operating system such as IBM's zSeries/Operating System (z/OS),MS-DOS, PC-DOS, MAC-iOS, WINDOWS, UNIX, OpenVMS, ANDROID, an operatingsystem based on LINUX, or any other appropriate operating system,including future operating systems.

In particular embodiments, the computing device includes the processor,the memory, the user interface, and the communication interface. Inparticular embodiments, the processor includes hardware for executinginstructions, such as those making up a computing device program. Thememory includes main memory for storing instructions such as computingdevice program(s) for the processor to execute, or data for processor tooperate on. The memory may include an HDD, a floppy disk drive, flashmemory, an optical disc, a magneto-optical disc, magnetic tape, aUniversal Serial Bus (USB) drive, a solid-state drive (SSD), or acombination of two or more of these. The memory may include removable ornon-removable (or fixed) media, where appropriate. The memory may beinternal or external to the computing device, where appropriate. Inparticular embodiments, the memory is non-volatile, solid-state memory.

The user interface is for displaying and interacting with communicationinterface. The user interface includes hardware, software, or bothproviding one or more interfaces for user communication with thecomputing device. As an example and not by way of limitation, the userinterface may include an input-output device, keyboard, keypad,microphone, monitor, mouse, printer, scanner, speaker, still camera,stylus, tablet, touchscreen, trackball, video camera, another userinterface or a combination of two or more of these.

The communications interface is for broadcasting to a virtualcollaboration platform over a network. The communication interfaceincludes hardware, software, or both providing one or more interfacesfor communication (e.g., packet-based communication) between thecomputing device and one or more other computing devices on one or morenetworks. As an example and not by way of limitation, communicationinterface may include a network interface controller (NIC) or networkadapter for communicating with an Ethernet or other wire-based networkor a wireless NIC (WNIC) or wireless adapter for communicating with awireless network, such as a WI-FI network. This disclosure contemplatesany suitable network and any suitable communication interface. As anexample and not by way of limitation, the computing device maycommunicate with an ad hoc network, a personal area network (PAN), alocal area network (LAN), a wide area network (WAN), a metropolitan areanetwork (MAN), or one or more portions of the Internet or a combinationof two or more of these. One or more portions of one or more of thesenetworks may be wired or wireless. As an example, the computing devicemay communicate with a wireless PAN (WPAN) (e.g., a BLUETOOTH WPAN), aWI-FI network, a WI-MAX network, a cellular telephone network (e.g., aGlobal System for Mobile Communications (GSM) network), or othersuitable wireless network or a combination of two or more of these. Thecomputing device may include any suitable communication interface forany of these networks, where appropriate.

A method and apparatus for retrieving accessing and storing medical datarelating to a patient during a medical procedure. The invention providesa single interface to many disparate forms of medical data which isaccessible over a local area network; wide area network directconnection or combinations thereof. In one embodiment an operating roomcontrol system for use during a medical procedure on a patient includesan input device a display device and a controller that is coupled to theinput device and the display device. The controller receives one or moreuser inputs transmits a command to a server located outside of theoperating room to retrieve medical data receives the medical data fromthe server and displays the medical data on the display device. Medicaldata can be captured by the controller using for example a camera and avideo/image capture board keyboard and microphone during surgery orexamination of the patient. The captured medical data can be stored onone or more remote servers as part of the patient records.

To assist a surgeon performing a medical procedure auxiliary imagesgenerally indicating internal details of an anatomic structure beingtreated are displayed and manipulated by the surgeon on a computerdisplay screen to supplement primary images generally of an externalview of the anatomic structure. A master input device controlling arobotic arm in a first mode may be switched by the surgeon to a secondmode in order to function instead as a mouse-like pointing device tofacilitate the surgeon performing such auxiliary information display andmanipulation.

The computer-based data processing system and method described above isfor purposes of example only, and may be implemented in any type ofcomputer system or programming or processing environment, or in acomputer program, alone or in conjunction with hardware. The presentinvention may also be implemented in software stored on acomputer-readable medium and executed as a computer program on a generalpurpose or special purpose computer. For clarity, only those aspects ofthe system germane to the invention are described, and product detailswell known in the art are omitted. For the same reason, the computerhardware is not described in further detail. It should thus beunderstood that the invention is not limited to any specific computerlanguage, program, or computer. It is further contemplated that thepresent invention may be run on a stand-alone computer system, or may berun from a server computer system that can be accessed by a plurality ofclient computer systems interconnected over an intranet network, or thatis accessible to clients over the Internet. In addition, manyembodiments of the present invention have application to a wide range ofindustries. To the extent the present application discloses a system,the method implemented by that system, as well as software stored on acomputer-readable medium and executed as a computer program to performthe method on a general purpose or special purpose computer, are withinthe scope of the present invention. Further, to the extent the presentapplication discloses a method, a system of apparatuses configured toimplement the method are within the scope of the present invention.

It should be understood, of course, that the foregoing relates toexemplary embodiments of the invention and that modifications may bemade without departing from the spirit and scope of the invention as setforth in the following claims.

What is claimed is:
 1. A system for providing clinical support for asurgery comprising: a processor; a display device configured to displayone or more real time images originating from a selected patient; and amemory comprising computing device-executable instructions that, whenexecuted by the processor, cause the processor to implement: acommunications interface for accessing a magnetic resonance imaging dataand an electronic health record data for the selected patient over anetwork, wherein said data is accessed but never stored; a userinterface for displaying and interacting with the communicationsinterface; and a generation module for broadcasting the user interfaceon the display device in such a way that the user interface isjuxtaposed to said one or more real time images, whereby the userinterface is never superimposed on the one or more real time images. 2.The system of claim 1, wherein the electronic health record datacomprises fusion biopsy data.
 3. The system of claim 1, wherein themagnetic resonance imaging data comprises a three-dimensional modelconfigured to be rotatable about a three-dimensional axis.
 4. The systemof claim 3, wherein the three-dimensional model configured to providezoom functionality and highlighting functionality.
 5. The system ofclaim 1, wherein the user interface is displayed along one screen of asplit-screen functionality generated by the generation module.
 6. Thesystem of claim 1, wherein the display device has an integratedbroadcasting device for receiving the user interface.
 7. The system ofclaim 1, wherein the magnetic resonance imaging data and the electronichealth record data comprise a preoperative imaging plan.
 8. The systemof claim 1, wherein the display device is a surgical console.
 9. Thesystem of claim 1, wherein the display device is a robotic console. 10.The system of claim 1, wherein the display device is a robotic consolefor guiding prostatic surgery.
 11. A system for providing clinicalsupport for a surgery comprising: a processor; a surgical consoleconfigured to display one or more real time images originating from aselected patient; and a memory comprising computing device-executableinstructions that, when executed by the processor, cause the processorto implement: a communications interface for accessing a magneticresonance imaging data and a fusion biopsy data set for the selectedpatient over a network, wherein said data is accessed but never stored,and wherein the magnetic resonance imaging data and the electronichealth record data comprise a preoperative imaging plan; a userinterface for displaying and interacting with the communicationsinterface; and a generation module for broadcasting the user interfaceon the surgical console in such a way that the user interface isjuxtaposed to said one or more real time images, wherein the magneticresonance imaging data comprises a three dimensional model configured tobe rotatable about a three-dimensional axis, and wherein thethree-dimensional model configured to provide zoom functionality andhighlighting functionality, whereby the user interface is neversuperimposed on the one or more real time images.
 12. The system ofclaim 11, wherein the user interface is displayed along one screen of asplit-screen functionality generated by the generation module.
 13. Thesystem of claim 11, wherein the display device has an integratedbroadcasting device for receiving the user interface.